Sound to haptic effect conversion system using mapping

ABSTRACT

A haptic conversion system is provided that intercepts audio data, such as a digital audio signal, analyzes the audio data in frequency, and divides the analyzed audio data into one or more audio frequency regions, where each audio frequency region includes one or more audio sub-signals. The haptic conversion system further maps the one or more audio frequency regions to one or more haptic frequency regions, where each haptic frequency region includes one or more haptic signals. The haptic conversion system further maps the one or more haptic effects to one or more actuators. The haptic conversion system further sends the one or more haptic signals to one or more actuators, in order to generate one or more haptic effects.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority of U.S. Provisional Patent ApplicationSer. No. 61/695,614, filed on Aug. 31, 2012, the disclosure of which ishereby incorporated by reference.

FIELD

One embodiment is directed generally to a device, and more particularly,to a device that produces haptic effects.

BACKGROUND

Haptics is a tactile and force feedback technology that takes advantageof a user's sense of touch by applying haptic feedback effects (i.e.,“haptic effects”), such as forces, vibrations, and motions, to the user.Devices, such as mobile devices, touchscreen devices, and personalcomputers, can be configured to generate haptic effects. In general,calls to embedded hardware capable of generating haptic effects (such asactuators) can be programmed within an operating system (“OS”) of thedevice. These calls specify which haptic effect to play. For example,when a user interacts with the device using, for example, a button,touchscreen, lever, joystick, wheel, or some other control, the OS ofthe device can send a play command through control circuitry to theembedded hardware. The embedded hardware then produces the appropriatehaptic effect.

Such devices can also be configured to play audio data, such as adigital audio signal. For example, such devices can include applicationsconfigured to play video data, such as a movie or video game, thatcontains an audio portion, or audio data, such as a song. Similar tohaptics, calls to additional embedded hardware capable of generatingaudio effects (such as speakers) can be programmed within the OS of thedevice. Thus, the OS of the device can send a play command throughcontrol circuitry to the additional embedded hardware, where theadditional embedded hardware then produces the appropriate audio effect.

SUMMARY

One embodiment is a system that converts an audio signal into one ormore haptic effects. The system analyzes the audio signal. The systemfurther divides the audio signal into one or more audio frequencyregions, where each audio frequency region includes one or more audiosub-signals. The system further maps the one or more audio frequencyregions to one or more haptic frequency regions, where each audiofrequency region is mapped to one or more corresponding haptic frequencyregions, and where each haptic frequency region includes one or morehaptic signals. The system further sends the one or more haptic signalsto an actuator to generate the one or more haptic effects.

BRIEF DESCRIPTION OF THE DRAWINGS

Further embodiments, details, advantages, and modifications will becomeapparent from the following detailed description of the preferredembodiments, which is to be taken in conjunction with the accompanyingdrawings.

FIG. 1 illustrates a block diagram of a haptic conversion system inaccordance with one embodiment of the invention.

FIG. 2 illustrates a block diagram of a haptic conversion system thatconverts an audio signal into one or more haptic signals that are sentto an actuator in order to generate one or more haptic effects,according to an embodiment of the invention.

FIG. 3 illustrates a block diagram of a haptic conversion system thatdivides an audio signal into one or more audio frequency regions, andmaps each audio frequency region to a haptic frequency region, accordingto an embodiment of the invention.

FIG. 4 illustrates a block diagram of a haptic conversion system thatmaps one or more haptic frequency regions to an actuator of a pluralityof actuators, according to an embodiment of the invention.

FIG. 5 illustrates a flow diagram of the functionality of a hapticconversion module, according to one embodiment of the invention.

DETAILED DESCRIPTION

One embodiment is a haptic conversion system that can receive audiodata, such as a digital audio signal, analyze the audio data infrequency, and divide the analyzed audio data into one or more audiofrequency regions, where each audio frequency region includes one ormore audio sub-signals. The haptic conversion system can further map theone or more audio frequency regions to one or more haptic frequencyregions, where each haptic frequency region includes one or more hapticsignals. The haptic conversion system can further map the one or morehaptic signals to one or more actuators. The haptic conversion systemcan further send the one or more haptic signals to one or moreactuators, in order to generate one or more haptic effects.

FIG. 1 illustrates a block diagram of a haptic conversion system 10 inaccordance with one embodiment of the invention. In one embodiment,system 10 is part of a mobile device, and system 10 provides a hapticconversion functionality for the mobile device. Although shown as asingle system, the functionality of system 10 can be implemented as adistributed system. System 10 includes a bus 12 or other communicationmechanism for communicating information, and a processor 22 coupled tobus 12 for processing information. Processor 22 may be any type ofgeneral or specific purpose processor. System 10 further includes amemory 14 for storing information and instructions to be executed byprocessor 22. Memory 14 can be comprised of any combination of randomaccess memory (“RAM”), read only memory (“ROM”), static storage such asa magnetic or optical disk, or any other type of computer-readablemedium.

A computer-readable medium may be any available medium that can beaccessed by processor 22 and may include both a volatile and nonvolatilemedium, a removable and non-removable medium, a communication medium,and a storage medium. A communication medium may include computerreadable instructions, data structures, program modules or other data ina modulated data signal such as a carrier wave or other transportmechanism, and may include any other form of an information deliverymedium known in the art. A storage medium may include RAM, flash memory,ROM, erasable programmable read-only memory (“EPROM”), electricallyerasable programmable read-only memory (“EEPROM”), registers, hard disk,a removable disk, a compact disk read-only memory (“CD-ROM”), or anyother form of a storage medium known in the art.

In one embodiment, memory 14 stores software modules that providefunctionality when executed by processor 22. The modules include anoperating system 15 that provides operating system functionality forsystem 10, as well as the rest of a mobile device in one embodiment. Themodules further include a haptic conversion module 16 that converts anaudio signal into one or more haptic signals that are used to produceone or more haptic effects, as disclosed in more detail below. Incertain embodiments, haptic conversion module 16 can comprise aplurality of modules, where each module provides specific individualfunctionality for converting an audio signal into one or more hapticsignals that are used to produce one or more haptic effects. System 10will typically include one or more additional application modules 18 toinclude additional functionality, such as Integrator™ software byImmersion Corporation.

System 10, in embodiments that transmit and/or receive data from remotesources, further includes a communication device 20, such as a networkinterface card, to provide mobile wireless network communication, suchas infrared, radio, Wi-Fi, or cellular network communication. In otherembodiments, communication device 20 provides a wired networkconnection, such as an Ethernet connection or a modem.

Processor 22 is further coupled via bus 12 to a display 24, such as aLiquid Crystal Display (“LCD”), for displaying a graphicalrepresentation or user interface to a user. The display 24 may be atouch-sensitive input device, such as a touch screen, configured to sendand receive signals from processor 22, and may be a multi-touch touchscreen.

System 10 further includes a plurality of actuators 26 (e.g., actuators26A and 26B). One of ordinary skill in the art would readily appreciatethat in the illustrated embodiment of FIG. 1, the plurality of actuators26 includes two actuators (i.e., actuators 26A and 26B), but that inalternate embodiments, the plurality of actuators 26 can include anynumber of actuators. Processor 22 may transmit a haptic signalassociated with a haptic effect to one or more actuators of theplurality of actuators 26, where each actuator of the one or moreactuators, in turn, outputs haptic effects. Each actuator of theplurality of actuators 26 may be, for example, an electric motor, anelectro-magnetic actuator, a voice coil, a shape memory alloy, anelectro-active polymer, a solenoid, an eccentric rotating mass motor(“ERM”), a linear resonant actuator (“LRA”), a piezoelectric actuator, ahigh bandwidth actuator, an electroactive polymer (“EAP”) actuator, anelectrostatic friction display, or an ultrasonic vibration generator.Furthermore, each actuator of the plurality of actuators 26 may be of adifferent actuator type.

In some embodiments, system 10 further includes one or more speakers 28.Processor 22 may transmit an audio signal to speaker 28, which in turnoutputs audio effects. Speaker 28 may be, for example, a dynamicloudspeaker, an electrodynamic loudspeaker, a piezoelectric loudspeaker,a magnetostrictive loudspeaker, an electrostatic loudspeaker, a ribbonand planar magnetic loudspeaker, a bending wave loudspeaker, a flatpanel loudspeaker, a heil air motion transducer, a plasma arc speaker,and a digital loudspeaker.

FIG. 2 illustrates a block diagram of a haptic conversion system thatconverts an audio signal into one or more haptic signals that are sentto an actuator in order to generate one or more haptic effects,according to an embodiment of the invention. According to an embodiment,a mechanism is provided for adjusting one or more parameters of anaudio-to-haptic conversion process based on various measures of an inputaudio signal and/or external data. From a user perspective, the resultis a haptic experience that is more selective in its response, enablingvibrotactile feedback for foreground events (such as collisions,explosions, etc. in games), and reducing or muting feedback forbackground events (such as in-game music or ambient noise). Bydistinguishing between perceptually important qualities of an audiosignal, the audio-to-haptic conversion process can prevent a “backgroundrumble” that can occur in games that have both sound effects and a musictrack, or media files that have significant transitions but also haveambient noise.

The illustrated embodiment of FIG. 2 illustrates the parameters of thehaptic conversion system being updated using the dashed lines. In oneembodiment, the haptic conversion system includes a real timemodification of parameters of the audio-to-haptic conversion process tostress certain characteristics of the audio file through haptics, wherethis is accomplished by modifying, in real time or during offlineconversion, the parameters of waveform and boombox modules of the hapticconversion system, as is further described in greater detail. Inaddition, a pre-process module may modify the audio signal itself usingaudio processing algorithms in order to enhance or modify the overallaudio-to-haptic conversion process. In an alternate embodiment, an audiosignal can be pre-processed to extract the appropriate information andthen stored back in the audio signal to be interpreted, in real time, bythe audio-to-haptic conversion process.

According to the embodiment, the haptic conversion system receives audiosignal 200. In certain embodiments, audio signal 200 includes one ormore digital audio buffers, such as pulse-code modulation (“PCM”) audiobuffers, where each PCM audio buffer comprises one or more PCM audiodata frames. In other embodiments, audio signal 200 includes one or morestructure-based audio buffers, such as Musical Instrument DigitalInterface (“MIDI”) audio buffers, where each MIDI audio buffer comprisesone or more MIDI audio data frames. In yet other embodiments, audiosignal 200 includes one or more frequency domain audio buffers, such asMPEG-2 Audio Layer III (“MP3”) audio buffers, where each MP3 audiobuffer comprises one or more MP3 audio data frames. In yet otherembodiments, audio signal 200 includes one or more audio buffers thatare of other audio formats known to one of ordinary skill in the art. Inyet other embodiments, audio signal 200 also contains haptic informationthat can include parameters or other cues used to configure theconversion algorithm.

The haptic conversion system subsequently sends audio signal 200 to apre-process module 210, which performs any pre-processing necessary onaudio signal 200. Such pre-processing can include under-sampling audiosignal 200. An example scenario where audio signal 200 is under-sampledis where audio signal 200 has a sampling frequency greater than 8kilohertz (“kHz”). Such pre-processing could alternatively includefusing an average of two or more channels of audio signal 200, whereaudio signal 200 is a stereo audio signal. Thus, the goal of pre-processmodule 210 is to ensure that an estimator module receives a consistentsignal, regardless of the format of audio signal 200. The pre-processingcan be based on one or more parameters. The haptic conversion systemsubsequently sends audio signal 200 to an estimator module 220, whichanalyzes audio signal 200 to determine one or more audio frequencyregions of audio signal 200. In certain embodiments, estimator module220 can determine a number of filters to use to determine one or moreaudio frequency regions of audio signal 200. Estimator module 220 canfurther adjust one or more parameters of pre-process module 210, and canfurther adjust one or more parameters of a boombox module (representedin FIG. 2 as boombox modules 240 and 241) and/or one or more parametersof a waveform module (represented in FIG. 2 as waveform module 250). Thehaptic conversion system subsequently sends audio signal 200 to one ormore filters (represented in FIG. 2 as filters 230, 231, and 232), whereaudio signal 200 is divided into the one or more determined audiofrequency regions. Further, as part of the sending of audio signal 200to one or more filters, each audio frequency region of audio signal 200is produced by a separate filter. The determination of the one or moreaudio frequency regions can be based on one or more parameters. Suchdivision is described in greater detail with respect to FIG. 3.

The haptic conversion system further sends certain audio frequencyregions of audio signal 200 to a boombox module (represented in FIG. 2as boombox modules 240 and 241), where the boombox module calculates amaximum value of each audio frequency region of the audio signal. Theboombox module further generates a haptic frequency region that includesone or more haptic signals based on the calculated maximum value, andmaps the audio frequency region of audio signal 200 to the generatedhaptic frequency region. The generation of the haptic frequency regioncan be based on one or more parameters. Such mapping is furtherdescribed in greater detail with respect to FIG. 3. The hapticconversion system further sends other audio frequency regions of audiosignal 200 to a waveform module (represented in FIG. 2 as waveformmodule 250), where the waveform module calculates a maximum value ofeach audio frequency region of audio signal 200, and further calculatesa sine carrier waveform based on the maximum value of the audiofrequency region of audio signal 200. The waveform module furthergenerates a haptic frequency region that includes one or more hapticsignals based on the calculated maximum value and the calculated sinecarrier waveform, and maps the audio frequency region of the audiosignal 200 to the generated haptic frequency region. The generation ofthe haptic frequency region can be based on one or more parameters. Suchmapping is also further described in greater detail with respect to FIG.3.

The one or more haptic signals of the one or more haptic frequencyregions are sent to a mixer module 260, where mixer module 260 combinesthe one or more haptic signals into a single combined haptic signal thatmay include additional filtering steps in order to prevent signalsaturation. In an alternate embodiment, mixer module 260 also maps the Ninput signals onto M output actuators taking into account the frequencyresponse, amplitude and physical location of each output actuator. Thecombined haptic signal is subsequently sent to a haptic effect playermodule 270, such as a TouchSense® 5000 Player module by ImmersionCorporation, where haptic effect player module 270 is configured to playone or more haptic effects at an actuator 280 by sending the combinedhaptic signal to actuator 280. Optionally, the combined haptic signal isalso stored within a haptic effect file 290, such as an ImmersionVibration Source (“IVS”) haptic effect file or an embedded track in astandard container format file, such as MPEG-4.

FIG. 3 illustrates a block diagram of a haptic conversion system thatdivides an audio signal into one or more audio frequency regions, andmaps each audio frequency region to a haptic frequency region, accordingto an embodiment of the invention. According to the embodiment, anaudio-to-haptic conversion process can include frequency mapping from anaudio frequency region to a haptic frequency region. The audio contentcan be analyzed in frequency, and then divided into one or more audiofrequency regions, where each audio frequency region includes one ormore audio sub-signals of the audio content. Further, each audiofrequency region can be mapped to one or more haptic frequency regions,where each haptic frequency region includes one or more haptic signals.Selection of frequencies for the haptic frequency regions can bedetermined based on actuator characteristics (for example, does theactuator have enough dynamic range at a specified frequency), bycharacteristics of haptic effects that can be perceived by a human, sothat the haptic frequency content is easily distinguishable, or by thefrequency content of the audio signal itself. For example, the powercontent in each audio frequency region can be extracted and used tomagnitude modulate the haptic content in each haptic frequency region.

According to the embodiment, the haptic conversion system receives audiosignal 310. In certain embodiments, audio signal 310 is similar to audiosignal 200 of FIG. 2. The haptic conversion system subsequently sendsaudio signal 310 to a pre-process module 320, which performs anypre-processing necessary on audio signal 310. Examples of pre-processingare previously described with respect to FIG. 2. The haptic conversionsystem subsequently sends audio signal 310 to an estimator module 330,which analyzes audio signal 310 to determine one or more audio frequencyregions of audio signal 310. Estimator module 330 subsequently dividesaudio signal 310 into the one or more determined audio frequency regions(represented in FIG. 3 as audio frequency regions 340, 341 and 342).Further, as part of the dividing audio signal 310 into the one or moredetermined audio frequency regions, each audio frequency region of audiosignal 310 is produced by a separate filter. Estimator module 330 canmodify or update this mapping in response to modifications in theprocessed audio signal. In certain embodiments, each audio frequencyregion represents a range of frequencies for sub-signals containedwithin audio signal 310. For example, in an embodiment where audiosignal 310 is divided into three audio frequency regions, a first audiofrequency region can include sub-signals with a frequency within a lowrange, a second audio frequency region can include sub-signals with afrequency within a middle range, and a third audio frequency region caninclude sub-signals with a frequency within a high range. This is merelyan example number of audio frequency regions, and an audio signal can bedivided into any number of audio frequency regions.

Subsequently, the one or more audio frequency regions are mapped to oneor more haptic frequency regions (represented in FIG. 3 as hapticfrequency regions 350, 351, and 352). In certain embodiments, eachhaptic frequency region represents a range of frequencies for one ormore haptic signals that are generated based on audio signal 310. Forexample, in an embodiment that includes three haptic frequency regions,a first haptic frequency region can include haptic signals with afrequency within a low range, a second haptic frequency region caninclude haptic signals with a frequency within a middle range, and athird haptic frequency region can include haptic signals with afrequency within a high range. This is merely an example number ofhaptic frequency regions, and there can be any number of hapticfrequency regions. In certain embodiments, filter 230 of FIG. 2corresponds to audio frequency range 340 and haptic frequency range 350of FIG. 3, filter 231 of FIG. 2 corresponds to audio frequency range 341and haptic frequency range 351 of FIG. 3, and filter 232 of FIG. 2corresponds to audio frequency range 342 and frequency range 352 of FIG.3.

In certain embodiments, audio frequency regions that include sub-signalswith a frequency in a lower range can be mapped to haptic frequencyregions that include haptic signals with a frequency in a lower range.Likewise, in these embodiments, audio frequency regions that includesub-signals with a frequency in a higher range can be mapped to hapticfrequency regions that include haptic signals with a frequency in ahigher range. In an example embodiment, the haptic conversion system candivide audio signal 300 into a plurality of audio frequency regions, andmap each audio frequency region to a corresponding haptic frequencyregion, as follows:

Audio Frequency Region Haptic Frequency Region  <200 Hz 100 Hz Between200 Hz and 4000 Hz 175 Hz >4000 Hz 250 Hz

The one or more haptic signals of the one or more haptic frequencyregions are sent to a mixer module 360, where mixer module 360 combinesthe one or more haptic signals into a single combined haptic signal. Thecombined haptic signal is subsequently sent to a haptic effect playermodule 370, such as a TouchSense® 5000 Player module by ImmersionCorporation, where haptic effect player module 370 is configured to playone or more haptic effects at an actuator 380 by sending the combinedhaptic signal to actuator 380. Optionally, the combined haptic signal isalso stored within a haptic effect file 390, such as an IVS hapticeffect file.

FIG. 4 illustrates a block diagram of a haptic conversion system thatmaps one or more haptic frequency regions to an actuator of a pluralityof actuators, according to an embodiment of the invention. It isgenerally known that humans can have difficulty discriminatingfrequencies, especially when those frequencies are close within acertain range. It is also generally known that some users prefer lowfrequency haptic effects generated by certain actuators, whereas otherusers prefer high frequency haptic effects generated by wide frequencyband actuators. According to the embodiment, a combination oflow-frequency response actuators (such as ERMs) can be combined withwide bandwidth actuators (such as piezoelectric actuators) to create aricher haptic content for applications, where the haptic content cannotbe created by a single actuator because of the dynamic and frequencycharacteristics of the actuator's design, as described in the followingtable:

ERM Piezoelectric Actuator Frequency Range 60-120 120-300 (Hz) MagnitudeCoupled to frequency; low Resonant at ~200 HZ; at low frequencies, highat independent of frequency high frequencies

Since the ERM can create low frequency vibrations (between 60 and 200Hz) but with fixed magnitude at each frequency, no superposition effectscan be created to output signals with low and high frequency content.The piezoelectric actuator, on the other hand, does not have the dynamicrange below 100 Hz to output meaningful haptic effects in this range. Acombination of the two actuators can provide a haptic effect designerwith a richer palette to choose from, enhancing the user experience.

Further, different algorithms can be created to combine the twoactuators. For example, in one instance, the piezoelectric actuator canbe used to display background music in a game, while the ERM can be usedto display interaction effects (such as explosions, collisions, etc.).The combination of the two actuators can also be used to create richertextures by combining the low and high frequency content.

According to the embodiment, the haptic conversion system receives audiosignal 410. In certain embodiments, audio signal 410 is similar to audiosignal 200 of FIG. 2 and audio signal 310 of FIG. 3. The hapticconversion system subsequently sends audio signal 410 to a pre-processmodule 420, which performs any pre-processing necessary on audio signal400. Examples of pre-processing are previously described with respect toFIG. 2. Pre-process module 420 also divides audio signal 410 into aplurality of audio regions (which are different from an audio frequencyregion, as the audio signal is divided into regions based oncorresponding actuators, rather than a frequency of sub-signals of theaudio signal), where each audio region corresponds to an actuator of aplurality of actuators (represented in FIG. 4 by actuators 470, 471, and472. In the illustrated embodiment, pre-process module 420 divides audiosignal 410 into three audio regions. However, this is merely an exampleembodiment, and pre-process module 420 can divide audio signal 410 intoany number of audio regions. In certain embodiments, the plurality ofactuators includes at least one low frequency actuator and one highfrequency actuator.

According to the embodiment, each audio region is divided into one ormore audio frequency regions, where each audio frequency regionrepresents a range of frequencies for sub-signals contained within thecorresponding audio region. In the illustrated embodiment, the firstaudio region is divided into audio frequency regions 430, the secondaudio region is divided into audio frequency regions 431, and the thirdaudio region is divided into audio frequency regions 432. Subsequently,for each audio region, the one or more audio frequency regions aremapped to one or more haptic frequency regions, where each hapticfrequency region represents a range of frequencies for one or morehaptic signals that are generated based on the audio region. In theillustrated embodiment, audio frequency regions 430 are mapped to hapticfrequency regions 440, audio frequency regions 431 are mapped to hapticfrequency regions 441, and audio frequency regions 432 are mapped tohaptic frequency regions 442.

Subsequently, for each audio region, the one or more haptic signals ofthe one or more haptic frequency regions are sent to a mixer module,where the mixer module combines the one or more haptic signals into asingle combined haptic signal. In the illustrated embodiment, the one ormore haptic signals of haptic frequency regions 440 are sent to mixermodule 450, where mixer module 450 combines the one or more hapticsignals into a first combined haptic signal. The one or more hapticsignals of haptic frequency regions 441 are sent to mixer module 451,where mixer module 451 combines the one or more haptic signals into asecond combined haptic signal. The one or more haptic signals of hapticfrequency regions 442 are sent to mixer module 452, where mixer module452 combines the one or more haptic signals into a third combined hapticsignal.

Each combined haptic signal is subsequently sent to a haptic effectplayer module, such as a TouchSense® 5000 Player module by ImmersionCorporation, where each haptic effect player module is configured toplay one or more haptic effects at a corresponding actuator by sendingthe respective combined signal to the respective actuator. In theillustrated embodiment, mixer module 450 sends the first combined hapticsignal to haptic effect player module 460, where haptic effect playermodule 460 is configured to play one or more haptic effects at actuator470 by sending the first combined haptic signal to actuator 470. Mixermodule 451 sends the second combined haptic signal to haptic effectplayer module 461, where haptic effect player module 461 is configuredto play one or more haptic effects at actuator 471 by sending the secondcombined haptic signal to actuator 471. Mixer module 452 sends the thirdcombined haptic signal to haptic effect player module 462, where hapticeffect player module 462 is configured to play one or more hapticeffects at actuator 472 by sending the third combined haptic signal toactuator 472.

FIG. 5 illustrates a flow diagram of the functionality of a hapticconversion module (such as haptic conversion module 16 of FIG. 1),according to one embodiment of the invention. In one embodiment, thefunctionality of FIG. 5 is implemented by software stored in memory oranother computer-readable or tangible medium, and executed by aprocessor. In other embodiments, the functionality may be performed byhardware (e.g., through the use of an application specific integratedcircuit (“ASIC”), a programmable gate array (“PGA”), a fieldprogrammable gate array (“FPGA”), etc.), or any combination of hardwareand software. Furthermore, in alternate embodiments, the functionalitymay be performed by hardware using analog components.

The flow begins and proceeds to 510. At 510, an audio signal isanalyzed. In certain embodiments, the audio signal is alsopre-processed. Also, in certain embodiments, one or more parameters usedto divide the audio signal into one or more audio frequency regions areadjusted. In some of the embodiments, one or more parameters used topre-process the audio signal are adjusted. The flow proceeds to 520. At520, the audio signal is divided into one or more audio frequencyregions, where each audio frequency region includes one or more audiosub-signals of the audio signal. The flow proceeds to 530. At 530, theone or more audio frequency regions are mapped to one or more hapticfrequency regions, where each audio frequency region is mapped to one ormore corresponding haptic frequency regions, and where each hapticfrequency region includes one or more haptic signals. The flow proceedsto 540. At 540, the one or more haptic signals of the one or more hapticfrequency regions are mapped to one or more actuators, where each hapticsignal is mapped to one or more corresponding actuators. The one or moreactuators can include at least one low frequency actuator and at leastone high frequency actuator. The flow proceeds to 550. At 550, the oneor more haptic signals are sent to one or more actuators to generate oneor more haptic effects, where each haptic signal is sent to itscorresponding actuator. The flow then ends. In certain embodiments, suchas where the audio signal includes a portion of a song, the flow can berepeated for multiple audio signals, where each audio signal is aseparate portion of the song. For example, a two-minute song could bedivided into a plurality of audio signals, where each audio signal is aduration of 17 milliseconds (“ms”), and the flow can be repeated foreach audio signal of the plurality of audio signals. This is only anexample, and the song could be of any duration, and each audio signal ofthe plurality of audio signals could be of any duration.

Thus, according to an embodiment, a haptic conversion system divides anaudio signal into one or more audio frequency regions, maps each audiofrequency region to one or more haptic frequency regions, and sends theone or more haptic signals of the one or more haptic frequency regionsto one or more actuators to generate one or more haptic effects. Theselection of frequencies within the haptic frequency region can causethe haptic conversion system to better utilize the features of the oneor more actuators (such as wide bandwidth actuators), and the use ofmore than one actuator can provide for a richer haptic experience.

The features, structures, or characteristics of the invention describedthroughout this specification may be combined in any suitable manner inone or more embodiments. For example, the usage of “one embodiment,”“some embodiments,” “certain embodiment,” “certain embodiments,” orother similar language, throughout this specification refers to the factthat a particular feature, structure, or characteristic described inconnection with the embodiment may be included in at least oneembodiment of the present invention. Thus, appearances of the phrases“one embodiment,” “some embodiments,” “a certain embodiment,” “certainembodiments,” or other similar language, throughout this specificationdo not necessarily all refer to the same group of embodiments, and thedescribed features, structures, or characteristics may be combined inany suitable manner in one or more embodiments.

One having ordinary skill in the art will readily understand that theinvention as discussed above may be practiced with steps in a differentorder, and/or with elements in configurations which are different thanthose which are disclosed. Therefore, although the invention has beendescribed based upon these preferred embodiments, it would be apparentto those of skill in the art that certain modifications, variations, andalternative constructions would be apparent, while remaining within thespirit and scope of the invention. In order to determine the metes andbounds of the invention, therefore, reference should be made to theappended claims.

We claim:
 1. A non-transitory computer-readable medium havinginstructions stored thereon that, when executed by a processor, causethe processor to convert an audio signal into one or more hapticeffects, the converting comprising: pre-processing the audio signal bymodifying the audio signal to create a modified audio signal andadjusting one or more parameters used to generate one or more hapticfrequency regions based on a measurement of the modified audio signal,wherein the adjusting configures the one or more parameters to enablehaptic feedback for foreground events and reduce or mute haptic feedbackfor background events; dividing the modified audio signal into aplurality of audio frequency regions that each comprise one or moreaudio sub-signals with frequencies within a corresponding audiofrequency range; generating the one or more haptic frequency regionsbased on the one or more parameters, wherein each of the one or morehaptic frequency regions comprises one or more haptic signals withfrequencies within a corresponding haptic frequency region; mapping eachof the plurality of audio frequency regions to at least onecorresponding haptic frequency region in the one or more hapticfrequency regions, wherein each audio frequency range is mapped to aplurality of corresponding haptic frequency ranges; and sending the oneor more haptic signals to an actuator to generate the one or more hapticeffects.
 2. The non-transitory computer-readable medium of claim 1, theconverting further comprising: mapping the one or more haptic signals toa plurality of actuators, wherein each haptic signal is mapped to one ormore corresponding actuators; and sending each haptic signal to itscorresponding actuator to generate the one or more haptic effects. 3.The non-transitory computer-readable medium of claim 2, wherein theplurality of actuators comprises a low frequency actuator and a highfrequency actuator.
 4. The non-transitory computer-readable medium ofclaim 1, wherein the modified audio signal is divided into the one ormore audio frequency regions using one or more filters.
 5. Thenon-transitory computer-readable medium of claim 4, wherein each audiofrequency region of the one or more audio frequency regions is producedby a separate filter of the one or more filters.
 6. The non-transitorycomputer-readable medium of claim 1, wherein a maximum value iscalculated for each audio frequency region of the one or more audiofrequency regions; wherein a sine carrier waveform is calculated foreach audio frequency region based on the maximum value; and wherein eachhaptic frequency region of the one or more haptic frequency regions isgenerated based on the maximum value and the sine carrier waveform. 7.The non-transitory computer-readable medium of claim 1, wherein aselection of one or more frequencies for each haptic frequency region ofthe one or more haptic frequency regions is determined based on one ormore characteristics of the actuator.
 8. The non-transitorycomputer-readable medium of claim 1, wherein the one or more audiofrequency regions comprise: a first audio frequency region thatcomprises one or more audio sub-signals with a frequency within a lowrange; a second audio frequency region that comprises one or more audiosub-signals with a frequency within a middle range; and a third audiofrequency region that comprises one or more sub-signals with a frequencywithin a high range.
 9. The non-transitory computer-readable medium ofclaim 8, wherein the one or more haptic frequency regions comprise: afirst haptic frequency region that comprises one or more haptic signalswith a frequency within a low range; a second haptic frequency regionthat comprises one or more haptic signals with a frequency within amiddle range; and a third haptic frequency region that comprises one ormore haptic signals with a frequency within a high range.
 10. Thenon-transitory computer-readable medium of claim 1, wherein the one ormore haptic signals are combined into a single combined haptic signal.11. The non-transitory computer-readable medium of claim 1, theconverting further comprising adjusting a plurality of parameters usedto divide the modified audio signal into the one or more audio frequencyregions.
 12. The non-transitory computer-readable medium of claim 1, theconverting further comprising adjusting a plurality of parameters usedto pre-process the audio signal.
 13. The non-transitorycomputer-readable medium of claim 1, wherein the modifying the audiosignal further comprises undersampling the audio signal.
 14. Thenon-transitory computer-readable medium of claim 1, wherein themodifying the audio signal further comprises fusing an average of two ormore channels of the audio signal.
 15. A computer-implemented method forconverting an audio signal into one or more haptic effects, comprising:pre-processing the audio signal by modifying the audio signal to createa modified audio signal and adjusting one or more parameters used togenerate one or more haptic frequency regions based on a measurement ofthe modified audio signal, wherein the adjusting configures the one ormore parameters to enable haptic feedback for foreground events andreduce or mute haptic feedback for background events; dividing themodified audio signal into a plurality of audio frequency regions thateach comprise one or more audio sub-signals with frequencies within acorresponding audio frequency range; generating the one or more hapticfrequency regions based on the one or more parameters, wherein each ofthe one or more haptic frequency regions comprises one or more hapticsignals with frequencies within a corresponding haptic frequency region;mapping each of the plurality of audio frequency regions to at least onecorresponding haptic region in the one or more haptic frequency regions,wherein each audio frequency range is mapped to a plurality ofcorresponding haptic frequency ranges; and sending the one or morehaptic signals to an actuator to generate the one or more hapticeffects.
 16. The computer-implemented method of claim 15, furthercomprising: mapping the one or more haptic signals to a plurality ofactuators, wherein each haptic signal is mapped to one or morecorresponding actuators; and sending each haptic signal to itscorresponding actuator to generate the one or more haptic effects. 17.The computer-implemented method of claim 15, wherein the plurality ofactuators comprises a low frequency actuator and a high frequencyactuator.
 18. The computer-implemented method of claim 15, wherein theone or more audio frequency regions comprise: a first audio frequencyregion that comprises one or more audio sub-signals with a frequencywithin a low range; a second audio frequency region that comprises oneor more audio sub-signals with a frequency within a middle range; and athird audio frequency region that comprises one or more sub-signals witha frequency within a high range.
 19. The computer-implemented method ofclaim 18, wherein the one or more haptic frequency regions comprise: afirst haptic frequency region that comprises one or more haptic signalswith a frequency within a low range; a second haptic frequency regionthat comprises one or more haptic signals with a frequency within amiddle range; and a third haptic frequency region that comprises one ormore haptic signals with a frequency within a high range.
 20. A hapticconversion system comprising: a memory configured to store a hapticconversion module; a processor configured to execute the hapticconversion module stored on the memory; and an actuator configured tooutput one or more haptic effects; wherein the haptic conversion moduleis configured to: pre-process the audio signal by modifying the audiosignal to create a modified audio signal and adjusting one or moreparameters used to generate one or more haptic frequency regions basedon a measurement of the modified audio signal, wherein the adjustingconfigures the one or more parameters to enable haptic feedback forforeground events and reduce or mute haptic feedback for backgroundevents; divide the modified audio signal into a plurality of audiofrequency regions that each comprise one or more audio sub-signals withfrequencies within a corresponding audio frequency range; generate theone or more haptic frequency regions based on the one or moreparameters, wherein each of the one or more haptic frequency regionscomprises one or more haptic signals with frequencies within acorresponding haptic frequency region; map each of the plurality ofaudio frequency regions to at least one corresponding haptic frequencyregion in the one or more haptic frequency regions, wherein each audiofrequency range is mapped to a plurality of corresponding hapticfrequency ranges; and send the one or more haptic signals to an actuatorto generate the one or more haptic effects.
 21. The haptic conversionsystem of claim 20, further comprising: a plurality of actuators, eachactuator of the plurality of actuators configured to output one or morehaptic effects; wherein the haptic conversion module is furtherconfigured to: map the one or more haptic signals to a plurality ofactuators, wherein each haptic signal is mapped to one or morecorresponding actuators; and send each haptic signal to itscorresponding actuator to generate the one or more haptic effects. 22.The haptic conversion system of claim 21, wherein the plurality ofactuators comprises a low frequency actuator and a high frequencyactuator.
 23. The haptic conversion system of claim 20, wherein the oneor more audio frequency regions comprise: a first audio frequency regionthat comprises one or more audio sub-signals with a frequency within alow range; a second audio frequency region that comprises one or moreaudio sub-signals with a frequency within a middle range; and a thirdaudio frequency region that comprises one or more sub-signals with afrequency within a high range.
 24. The haptic conversion system of claim23, wherein the one or more haptic frequency regions comprise: a firsthaptic frequency region that comprises one or more haptic signals with afrequency within a low range; a second haptic frequency region thatcomprises one or more haptic signals with a frequency within a middlerange; and a third haptic frequency region that comprises one or morehaptic signals with a frequency within a high range.